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ABSTRACT 

Introduction: Sleep disordered breathing in the 
paediatric population can manifest as an array of 
different systemic symptoms; among them is a distinct 
malocclusion and craniofacial phenotype. Emerging 
research suggests that the treatment of this 
malocclusion and/or craniofacial phenotype through 
orthodontic intervention may help with the symptoms 
of these patients. Selecting the patients who would 
benefit from orthodontic treatment can be a difficult 
task for the physician with minimal dental training. 
Therefore the aim of this study is to develop a simple 
index to be used by medical professionals to identify 
those paediatric patients with orthodontic treatment 
needs who may benefit their obstructive sleep apnoea 
(OSA) symptoms. 

Methods and analysis: The methodology in this 
project has been devised through the WHO'S 
recommendations on developing an index, with 
modifications based on the specific needs of this study. 
Based on the available literature, a draft index will be 
produced and subjected to multiple iterative revisions 
based on the feedback from: the Index Development 
Group, a group of multidisciplinary and internationally 
acclaimed experts in the field; the External Review 
Group, a group of potential end users and interested 
parties and the Steering Committee. Once the index has 
been formalised, it will be subjected to a pair of 
reliability tests using physicians and orthodontists 
scored 2 weeks apart. Subsequently, the index will be 
validated using dichotomous responses from 
orthodontists on whether they would treat a patient for 
OSA symptoms, and comparing the responses to the 
score of the index on the same patient. 
Ethics and dissemination: The index will be 
translated into French and will be presented in 
orthodontic and medical conferences, workshops, 
seminars, round table discussions, and free copies for 
download will be made available on the website of the 
University of Alberta Interdisciplinary Airway Research 
Clinic (iarc.ualberta.ca). Furthermore, the index will be 



Strengths and limitations of this study 



■ This index will help physicians and other medical 
professionals understand and identify which 
patients with obstructive sleep apnoea with mal- 
occlusions and craniofacial phenotypes are likely 
to benefit from targeted orthodontic treatment 
and will allow them to refer these patients 
accordingly. 

■ This index development acknowledges the multi- 
factorial nature of sleep disorder breathing (SDB) 
and the need for multidisciplinary care. The ideal 
end result of this index is to facilitate and 
enhance effective collaboration between invested 
dental and medical specialties. 

■ Development of a validated index will facilitate 
future epidemiology studies, allow for quality 
assurance and guide funding allocation. It will 
also allow long-term follow-up and audit in order 
to enter into comparisons with other centres. 



published in a peer-reviewed medical journal to further 
increase the exposure of the index. 



INTRODUCTION 

Snoring, although ubiquitous in the adult 
population, is considered abnormal in chil- 
dren and adolescents. 1 More importantly, it 
may serve as an indicator of a more severe 
respiratory problem that presents as a con- 
tinuum, from primary snoring to obstructive 
sleep apnoea (OSA). Reports vary on the 
prevalence of OSA, ranging from 0.7% to 5% 
of the population under 18 years old. 2-8 
Moreover, breathing-induced sleep disorders 
have been proven to have a profound effect on 
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the child's behaviour, growth and development; the 
myriad of symptoms include: morning tension-type head- 
aches, excessive morning thirst, excessive fatigue and 
sleepiness, abnormal shyness, withdrawn and depressive 
presentation, pattern of attention-deficit/hyperactivity dis- 
order, memory impairments, aggressiveness and irritability, 
among many others. 9-13 Other physiological processes 
that can be affected include stunted growth, 14 high blood 
pressure, 15 damage to the heart: ventricular hypertrophy, 16 
and cor pulmonale. 17 In addition to behavioural and sys- 
temic health consequences, craniofacial development is 
also affected. These patients generally have a craniofacial 
component contributing to their OSA, which would mani- 
fest as a retrognathic maxilla 18 or mandible, a long lower 
face height and restriction in the space of the upper 
airway. 19 20 Furthermore, when evaluating the polysomno- 
graphy of these patients, the evidence suggests that palatal 
expansion and mandibular advancement appliances 21-26 
can be of benefit at reducing the severity of OSA. Reverse 
pull headgear, 27-32 and maxillary and mandibular 
advancement surgery 33 have also been shown to have 
great promise at helping this group of patients. Since 
orthodontic treatment of the OSA craniofacial phenotype 
is an integral component to multidisciplinary care, it is 
essential for medical professionals (physicians, nurses, 
etc) to recognise the phenotype that would benefit from 
orthodontic treatment. Unfortunately, there are no guide- 
lines for non-dental trained practitioners to help identify 
which children with sleep disorder breathing (SDB) would 
benefit from orthodontic treatment. Therefore, this study 
aims to develop an index that can summarise the need for 
orthodontic treatment, in select cases of children with 
OSA, to physicians and adjunct medical professionals. 
Once the index is developed, it will be assessed for reliabil- 
ity and will be validated. On completion, this index will 
equip medical professionals with a simple way to assess 
which patients have malocclusions that contribute to their 
OSA and may benefit from orthodontic treatment. 

METHODS AND ANALYSIS 

This research is classified as an applied interdisciplinary 
medical research. The overall study design involves com- 
ponents of qualitative analysis and quantitative analysis. 
The qualitative components comprise focus groups and 
committee meetings. The quantitative components com- 
prise reliability tests and a cross-sectional validity test. 

Initial development 

In accordance with the WHO's recommendations on 
developing an index, 34 development of the index will be 
achieved through the following objectives: 

1. Establishing a Steering Committee; 

2. Scoping the index; 

3. Reviewing the literature; 

4. Drafting the index; 

5. Organising an Index Development Group (IDG); 

6. Organising an External Review Group (ERG) . 



Steering Committee 

The Steering Committee will be established a priori, and 
will be identified by the principal investigator based on 
the required expertise in the involved fields and based 
on availability and access. The Steering Committee will 
be responsible for overseeing every aspect of the study. It 
will be composed of a representative group of three 
experts in orthodontics, paediatric sleep medicine and a 
methodologist specialising in psychometric property ana- 
lysis. Their responsibilities will include: scoping the 
index, overseeing evidence retrieval, drafting the index, 
selecting members of the IDG and ERG, and finalising 
the index. In these meetings, unanimous consensus 
through discussion must be reached on all decisions 
before proceeding to the next step. 

Scoping the index 

Scoping is the process of defining what factors will be 
investigated in the literature for inclusion in the index. 
Scoping will be achieved through the combined experi- 
ence and expertise of the Steering Committee, and each 
factor that is suggested will be further investigated in the 
literature to establish an evidence-based approach to the 
development of the index. 

Reviewing the literature 

For each of the scoped factors, the literature will be 
reviewed to establish relevance. Specifically, the evidence 
must demonstrate that appropriate treatment of the cra- 
niofacial factor in question will lead to an improvement 
in the OSA symptoms. The literature search will not be 
a systematic review, and will include results from 
PubMed, MEDLINE, EMBASE, Web of Science, Scopus, 
the Cochrane Library and grey literature. The search 
strategy, including inclusion criteria, exclusion criteria 
and key words, will be devised through the advice of the 
Steering Committee. The searches themselves will be 
done in duplicate, using two different assessors. Of the 
relevant articles, hand searches will be performed on 
the bibliography lists. The results will be discussed and 
consensus will be reached between both assessors for the 
results of the search. Priority will be given to the results 
of well-conducted and well-reported systematic reviews 
and randomised control trials. Each of the factors will 
be assessed on its effect on paediatric OSA. 
Furthermore, the literature will be searched for the cra- 
niofacial morphology of paediatric patients with OSA. 
All of this information will be brought back to the 
Steering Committee for discussion of inclusion/elimin- 
ation of factors in the index. 

Drafting the index 

Since this index will have visual-rating scales for each of 
the craniofacial and occlusal factors, the Steering 
Committee will devise an outline suitable for displaying 
each of the factors representing the index. The factors 
in the drafted index will also have a number of levels 
that will divide the factor into categories of severity. The 
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number of levels for a given factor will also be deter- 
mined by the Steering Committee. Details describing 
the nature of the illustrations, the amount of levels that 
each of the factors will have and the general layout of 
the index will be agreed on by the Steering Committee, 
and the graphic artist will then design a preliminary 
version of the index based on the relevant factors and 
the feedback from the Steering Committee. 

The index development meeting 

The IDG is formed by a group of external multidisciplin- 
ary experts who provide evidence-based recommenda- 
tions on the content, layout and development process of 
the index. This eclectic group will be chosen by the 
Steering Committee based on experience in the field, 
expertise and ability to contribute, availability and 
access. The group should be small enough to be able to 
have effective discussions, yet large enough to ensure 
appropriate representation from all stakeholders. In this 
study, we plan on including a broad spectrum of 
medical specialists, family physicians, orthodontists and 
methodologists. The goals of this meeting are to gain 
additional information to strengthen the index, as well 
as to gain the professional credibility of a broad range of 
experts on the topic of paediatric OSA. 

Once members have been chosen by the Steering 
Committee and have accepted the invitation to partici- 
pate, the IDG members will be emailed a document 
4 days prior to the meeting, explaining the purpose of 
the index and meeting, how the meeting will proceed, 
what to expect in the meeting and a brief literature 
review of the orthodontic techniques currently available 
to help with the symptoms of OSA. Moreover, every 
member will be randomly assigned a number in order 
to maintain anonymity of the responses. 

Procedure of the IDG meeting 

The meeting will start with a brief introduction summar- 
ising the literature review, the purpose of the index and 
its relevance. The meeting will then proceed to collect 
feedback (see online supplementary appendix 1) on the 
aforementioned chosen factors using a modified Delphi 
technique, 35 36 which is a communication technique 
that structures the meeting and minimises bias in 
responses. Participants' responses will be collected 
through a web-based response portal. Each factor will be 
explored through yes or no questions, and yes responses 
followed up with a scale of 1-9 based on its importance 
for decision-making. Once all the feedback is received 
on a particular factor, the summary of the results will be 
displayed for everyone to see. Discussion will ensue for a 
maximum of 5 min, with each person talking no more 
than 1 min. Then everyone will be asked to re-enter 
their feedback on the website for the same factor in 
light of the discussion. This cycle will continue until a 
consensus of greater than 80% of the members is 
reached. Once consensus is reached we will move on to 



the next factor and the participants will be asked to give 
feedback on this new factor in a similar manner. 

In addition, two negative control factors will be used 
to calibrate the responses. The first negative control will 
be of orthodontic relevance but will have no effect on 
the amelioration of the symptoms of OSA (ie, crowding 
or impacted canine). The second negative control will 
be of relevance to OSA symptoms, but cannot be 
changed by orthodontic treatment (ie, body mass index 
or neck circumference). 

Once all the factors have been discussed and consen- 
sus reached, we will get feedback through the website on 
the: alignment — importance of the index and whether 
the development process is appropriate; relevance — 
content analysis and whether all the factors identified 
are important; representation — if there is anything that 
needs to be added to the index (see online 
supplementary appendix 2). Finally, additional written 
feedback will be accepted at the very end of the session. 

All the feedback from the IDM will be summarised 
and presented to the Steering Committee, and decisions 
will be made to remove factors, modify factors and/or 
modify the outline and layout of the index. These modi- 
fications will subsequently be presented to the graphic 
artist and a second draft will be procured. 

The ERG meeting 

The ERG is composed of end users and interested 
parties. The group will also be identified based on the 
advice of the Steering Committee based on the ability of 
the group to assess the index and contribute to its devel- 
opment, availability and access. This group is not respon- 
sible for any content analysis, instead it will be 
responsible for reviewing the layout, simplicity and ease 
of use. It will also be responsible for assessing the useful- 
ness of the index in the healthcare setting and to give 
feedback on the feasibility of implementing the index in 
practice. The goal of this meeting is to gain end user 
approval for the ease and feasibility of administering the 
index. This group should be large enough to be a repre- 
sentative sample of the population, yet small enough to 
allow for ease of explanation and healthy discussion. It is 
not as structured as the IDM, and allows for the partici- 
pants to freely express their opinions in an open forum. 

In this meeting, we will explain the theory behind the 
index by briefly reviewing the literature and then 
explain the purpose of the index. Subsequently we will 
show the group a pilot version of the index and a paper 
will be distributed to receive their feedback based on 
the following questions: 

1. Do you understand the purpose of the index? 

2. Do you understand what each factor is assessing? 

A. If not, which one(s) do you not understand and 
why? 

3. On a scale of 1-10, how simple would you rate this 
index to understand and use? 

4. Would you use this index in your practice? 

5. Other recommendations. 
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By compiling the comments and scores onto a table, 
the results will display the feedback from the ERG 
meeting. All these results will be presented to the 
Steering Committee, and decisions will be made to 
modify the outline and layout of the index. These modi- 
fications will subsequently be presented to the graphic 
artist and a third draft will be procured. 

Reliability 

Reliability of the index will be tested within a group that 
represents the typical end user population. This includes 
physicians of family medicine, paediatricians, paediatric 
ENT or paediatric sleep physicians. Reliability will also 
be assessed within a group of orthodontists who, by their 
training, are experts at assessing malocclusion and cra- 
niofacial morphology. Therefore 10-20 physicians and 
orthodontists will be recruited as examiners in this study. 

A pool of 15-40 randomly selected patient charts from 
the University of Alberta Inter-disciplinary Airway 
Research Clinic (I-ARC) will be recruited as reliability 
test participants, and their intraoral and extraoral photo- 
graphs will be used in the reliability assessment. After a 
brief explanation about the use and application of the 
index, the physicians as well as the orthodontists will 
apply the developed index to the sample patient pool's 
pictures once. In order to diminish recall bias, applica- 
tion of the index will be repeated 2 weeks later. 

Intra-rater reliability and inter-rater reliability between 
the physicians, between the orthodontists, and between 
the physicians and orthodontists will be compared. The 
reliability will be assessed using interclass correlation 
coefficients and Bland-Altman plots. 

Validation 

The index will be validated using dichotomous 
responses from orthodontists on whether specific 
patients would require orthodontic treatment to help 
their OSA symptoms and comparing them to the score 
that the index gave those same patients. This will be 
achieved by setting up a website where 30 orthodontists 
with experience in dealing with paediatric OSA will be 
recruited to take the assessment. The website will 
contain extraoral and intraoral pictures of patients ran- 
domly selected from the Interdisciplinary Airway 
Research Clinic diagnosed with OSA, and the orthodon- 
tists will be asked to rate these patients, using a 'y es ' or 
'no' response, on whether they would benefit from 
orthodontic treatment for their OSA symptoms. The 
index will then be applied on the same patients by the 
principal investigator and the score of each patient will 
be recorded. Using a stepwise multiple logistic regres- 
sion, each of the identified factors will be given a weight; 
this will represent the relative importance of the factor. 
Once analysed, if the correlation is high between the 
expert scores and the cluster groups then the clusters 
are meaningful and valid. Furthermore, a cut-off for 
most efficient score above which to refer will be chosen 
using a graph and observing the value that optimises the 



sensitivity, specificity and overall accuracy of the index. 
Finally, four grades of treatment need will be deter- 
mined using the 25th centile ranges. The grades will be: 

1. Minimal need; 

2. Mild need; 

3. Moderate need; 

4. Severe need. 



DISCUSSION 

The development plan of this index has been conceived 
through a modification of the WHO handbook for 
guideline development, 34 as well as reviewing the ortho- 
dontic literature for ways indices have been previously 
developed. The WHO provided an excellent starting 
point; from there modifications were made to cater to 
the specifics of this study, given that there are differ- 
ences between developing a guideline and an index for 
orthodontic treatment need. The literature was useful, 
and among the index development protocols reviewed, 
some assessed the orthodontic treatment need within 
the entire population, 37-40 while others assessed it for a 
given subpopulation 41-46 ; each had strengths and weak- 
nesses, and thus we further modified our methods, 
synthesising a protocol for our particular needs from the 
available literature and using the experience and expert- 
ise of the authors. Through this protocol we aim to 
develop an index that fulfils all of the following 

• 47 

criteria : 

1. Gradient of numeric values: the severity of the 
orthodontic treatment need within the paediatric 
patients with OSA should be defined within a 
numerical scheme that demonstrates a finite and 
progressive gradient from low need to high need. 

2. Equal sensitivity: should demonstrate equal sensitiv- 
ity throughout the scale. 

3. Clinical importance: the numerical scale should cor- 
respond with the clinically appraised orthodontic 
treatment need of paediatric patients with OSA. 

4. Statistical ease: should be amendable to statistical 
analysis. 

5. Reliability: should have a high intra-rater and inter- 
rater reliability. 

6. Practical: the instruments required to score the 
index should be practical to the setting in which it 
will be administered. 

7. Minimal judgement: applying the index should 
require minimal judgement. 

8. Simple: the index should not have a high financial 
or time cost, and thereby should be simple enough 
to administer to many patients. 

9. Detect change: The index should be able to detect 
changes in orthodontic treatment need in paediat- 
ric patients with OSA. 

10. Validity: should be valid over time. 

Validity can be characterised into different types: face 
validity, content validity, construct validity and criterion 
validity. 48 In this study, we will examine these kinds of 
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validity at different stages of development. The first draft 
of the index will focus on establishing face validity. 
Feedback from the Steering Committee and IDG will 
assist in establishing content and construct validity. 
Assuming that the 'gold standard' in assessing the ortho- 
dontic treatment need in paediatric patients with OSA is 
an orthodontist with experience in dealing with paediat- 
ric patients with OSA, then the subsequent modification 
of the index based on the reliability tests and the dichot- 
omous responses from the orthodontists provides the 
index with the necessary criterion-related validity evi- 
dence through statistical means. 49 

Significance 

This index will help physicians and other medical pro- 
fessionals identify which craniofacial phenotypes may 
benefit from orthodontic treatment as part of their 
multidisciplinary OSA management. Furthermore, due 
to the diverse medical effects of sleep deprivation, there 
will be a trend to make sleep apnoea into a centralised 
service, where the main focus is for a highly trained 
multidisciplinary team to treat a high volume of patients 
to a standardised protocol, where meticulous documen- 
tation is exercised. This index is part of that documenta- 
tion process. It will allow for quality assurance, funding 
allocation and epidemiological studies to be performed. 
It will also allow long-term follow-up and audit in order 
to enter into comparisons with other centres. 



Dissemination plan 

The dissemination of this index will be performed 
through a variety of ways in order to maximise its reach. 
Primarily, it will be published in a peer-reviewed journal, 
which will allow its introduction to the scientific literature. 
The journal should be a respected medical journal with 
broad reach, in order to allow the greatest number of phy- 
sicians to be exposed to the index. Subsequently, it will be 
translated into French, in order for it to be accessible to 
the entire Canadian and American population of medical 
professionals. Moreover, the index will be presented at 
national and international conferences to increase the 
awareness of the index among the scientific community. 
Finally, the index will be used at the University of Alberta's 
Interdisciplinary Airway Research Center, and more 
research will be conducted, so that future research in this 
centre will incorporate it. It will also be placed on the 
University of Alberta's Interdisciplinary Airway Research 
Center's website under the physician section, to further 
educate doctors who visit the site on the index. 
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